Context

The test infrastructure consists of the facilities and resources necessary to carry out the testing satisfactorily. A distinction is made between the facilities for test execution (test environments), for supporting the testing (test tools) and for the day-to-day work of the testers (workplaces).

The setup and maintenance of infrastructure involves specific expertise. It is something that testers in general have limited knowledge of, but upon which they nevertheless are very dependent (without infrastructure, there can be no test). All the responsibilities surrounding the setting up and maintaining of infrastructure are therefore often given to a separate maintenance department, necessitating close co-operation with these other (sometimes external) parties during the test. This means that test managers land in a situation where they have no authority over the setup and maintenance of the infrastructure (the maintaining party has the say-so), while they nevertheless depend on it. This can lead to conflict. For example, the situation could arise in which this maintaining party gives priority to solving production-disrupting problems above solving problems in a test environment. Furthermore, a maintenance department often also has particular security guidelines (e.g. authorisation checks, fixed backup times, installation procedures) that cannot easily be ignored. This is something that should be taken account of during the testing and, with that, the responsibility for the setup and maintenance of the infrastructure is an important area of focus for the test manager. A means of alleviating the concern for this support process is the permanent test organisation, which will take full responsibility for the setup and maintenance of the test infrastructure.

With a test project, it is important to pay special attention to the setup and maintenance of the infrastructure. In order to keep the focus on this during the test, there is a separate phase within the TMap life cycle model. It is a phase that runs parallel with the phases of Preparation, Specification, Execution and Completion. For some activities, there are dependencies between these and activities in the other TMap phases.

Test environment

A suitable test environment is required for the dynamic testing of a test object (the running of software).

Hardware refers to all the tangible parts of a computer (screen, hard disk, network card, et cetera). Test environment software refers to all the programs that should be present on the available hardware in order to run the software under test, such as operating programs, DBMS, network and other support programs. Connections are everything that is required to allow the test object to communicate with other systems. The environment data is the set of data that the test environment requires to be able to work with these (user profiles, network addresses, root tables, et cetera). Maintenance tools are tools that are required specifi cally to keep the test environment operational, and management processes are all the activities that are carried out around the setup and maintenance of a test environment.

Test tools

Test tools can be used as instruments for achieving higher productivity and/or effectivity on the part of the testers and the testing. With the use of test tools, the emphasis is on “support” (see the definition). This means that a test tool is only a tool if the use of it delivers something; using a tool should not be a goal in itself.

One of the conditions for the successful use of test tools is the presence of a structured test approach. In a well-managed process, tools can certainly deliver significant added value, but they are counterproductive in an inadequately managed test process. The reason for this is that automation (what test tools actually do) requires a certain repeatability and standardisation of the activities to be supported. An unstructured process cannot meet these conditions. However, the deployment of test tools can function as leverage for implementing a structured approach. Structuring and automation should therefore go hand in hand, in short: “Structure and Tool”.

Workplaces

One of the aspects that is often forgotten in testing is the provision of a workplace, where testers can perform their tasks effectively and efficiently in satisfactory conditions. This involves an office setup in the most general sense, and so the workplace consists of more than just office space and a PC. Issues, too, such as e.g. entry passes, power supply and lunch-break facilities all have to be arranged.

If the testing is carried out in the framework of a project, extra office space should be organised. It is advisable to bring the test team together in one location (a room or a floor). This will form a basis for good mutual cooperation and coordination within the team. If that is not possible, the location of team members in various rooms should correspond with, for example, the allocation of the various system parts to the testers, the test types to be applied, et cetera. If developer and tester work together in multidisciplinary teams, they should be situated together in one location.

As with every project activity, a great deal of consultation takes place in testing. Because testing finds itself at the crossroads of the various activities in the project, testers have a lot of contact with the various groups (such as designers, programmers, administrators and users). It is advisable to place the test team in the vicinity of these groups. There are examples of improved test processes thanks to the relocation of the test team to the physical ‘middle’ of the project organisation. This resulted in, among other things, increased mutual respect between the testers and other project participants, which benefited the quality.

The workplace intended for a tester at first sight does not differ much from the standard workplace. But appearances are deceiving. What is being tested is often new to the organisation and the workplace. Testers may find themselves in a situation in which their workplace is unprepared for the new software. It is therefore often necessary to arrange separate authorisations for testers. For example, testers should have the possibility of installing the new software on their local PC, and this may also be necessary in order to use particular test tools.

Preconditions

Before the setup and maintenance of infrastructure phase can be started, the description of the required infrastructure at an overall level, including the general plan, should be known and established in the test plan and/or master test plan. If test tools are being used, it should be known how the various activities within TMap are to be performed.

Method of operation

On the basis of the definition of the infrastructure set out in the test plan, it is considered whether closer specification and more detail are necessary. Besides the description of the required resources, it is also described what is expected of the suppliers during the maintenance of these resources. Since different expertise is required, the realisation of the infrastructure is often carried out by other parties. From within the test project, the progress of the realisation is monitored and if the progress is threatened, actions are devised. The realisation should be completed before the Execution phase begins, but preferably earlier. Simultaneously with the realisation of the infrastructure, a checklist is created that includes specifi c checks. This is used to determine, upon delivery, whether the infrastructure supplied meets the previously set requirements. After delivery, the infrastructure should be kept available for the testers at the quality level determined at the start of the phase. At the end of the test assignment, it is examined which parts of the infrastructure should be preserved. These can then be reused in future (re)tests.

Roles / responsibilities

It is advisable to delegate the organisation of this phase to someone other than the test manager. This individual then takes the role of test infrastructure coordinator.

Activities

The basis of the Setting up and maintaining infrastructure phase is defined in the Planning phase. Here, within the activity “Defining the infrastructure” the infrastructure required at overall level is described, including the planning. This description (from the master test plan or test plan), serves as input for the first activity in this phase.

The Setting up and maintaining infrastructure phase consists of the following six activities:

1. Specifying the infrastructure
2. Realising the infrastructure
3. Specifying the infrastructure intake
4. Intake of the infrastructure
5. Maintaining the infrastructure
6. Preserving the infrastructure

Figure 1 - "Setting up and maintaining infrastructure” indicates the sequence and dependency between the various activities. Activities “Realising the infrastructure” and “Specifying the infrastructure intake” can be carried out in parallel. The dependency between the end of activity “Intake of the infrastructure” and the start of the Execution phase is significant. Before the test execution can start, there must be a correctly operating test infrastructure.

That is why it is essential to plan activity “Intake of the infrastructure” before the start of the test execution. It is even advisable to plan this well in advance (and the preceding activities as well) in order to prevent any start up problems with the test infrastructure from causing the test execution to overrun. Test execution often finds itself on the critical path of the entire project, and so problems with the test infrastructure indirectly cause the project to overrun. Also, an operational infrastructure is very handy in the Specification phase. Test scripts can be tried out, and test data (in files, for example) implemented.

Figure 1 - "Setting up and maintaining infrastructure”

In the definition of test infrastructure, it says that a distinction is made between test environment, test tool and workplace. For each of these three, the activities model, as previously described, should be followed. The activities of the three parts have a mutual relationship as regards timeliness. Activity “Intake of the infrastructure” plays an important role here. The intake of the infrastructure forms the link with the other phases in TMap and is also the common link between the three parts.

It is advisable to organise the workplace as quickly as possible, and it should be ready before the testers arrive. This means it should be prepared during the Planning phase. Often it is even necessary to have the workplace operational before the intake of the test environment can begin. And, in turn, the test environment often needs to be operational before a start can be made on the intake of the test tool. This is made clear in figure 1 above.The setup and maintenance of the infrastructure is a very complex operation, with many internal and external dependencies. The organising demands close attention and it is therefore advisable to arrange this with the test infrastructure coordinator.